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INTRODUCTION 



The Association celebrated its Thirtieth Anniversary Forum in Louisville with the 
theme, "Institutional Research-Coming of Age." The Louisville theme and program 
attempted to review and further develop the three decades of the profession by 
linking together the emphasis on leadership and professional standards begun at the 
Phoenix Forum of 1988 with the press of social and technological challenges raised 
last year in Baltimore. 

The Forum Program with over one-hundred twenty-five panels, contributed papers, 
demonstrations and other professional development opportunities provided 
participants with a wide array of issues and experiences. Continuing the practice of 
recent years, the program was again structured around seven basic tracks, although 
the topical thrusts of se.cral t nicks were expanded, and some sessions tended to 
span beyond a single topical track. 

The synopses of the four general sessions are presented in this volume. In the 
opening session, Robert Taylor challenged in an unusual way institutional 
researchers to play a greater leadership and team building role in their institutions. 
In another session, George Keller provided a future vision of the opportunities and 
expectations for institutional researchers. Bob Pace shared his personal 
retrospective on the development of the profession, while Frans van Vught challenged 
institutional researchers to develop a scientific base for the profession. In this year's 
presidential session, Gerald McLaughlin reminded the membership that we have 
responsibility for providing institutional leadership and research on the critical issues 
confronting institutions, while Larry Sherr expanded on his vision presented last 
year on the need to achieve higher quality programs and services in our colleges and 
universities. 

The synopses contained in this volume provide a glimpse into the key ideas and view 
of the profession which guided the review of its past and challenges to its future. I 
am confident that the discourse begun in Louisville on institutional research as a 
profession coming of age will continue in the years ahead. 




Louisville Forum Chair 



Leadership and Team Building: Key Ingredients to the Institutional Research Role of the 
Future 

(opening general session) 



Robert L Taylor 

Dean, School of Business 
University of Louisville 



(This is an edited record of an address presented in the Grand Ballroom of the Gait House, 
in Louisville, Kentucky, on Sunday evening, May 13, 1990.) 




Setting the Stage 

I would like to work with you in thinking through (1) a philosophy 
about leadership and team building, (2) the role that institutional 
research plays now in university settings, and (3) what must 
change for institutional research to play a significant role in the 
future. This exercise is not another tool or technique, but 
something I'd like you to just spend some time thinking about 

My experience indicates that we spend a lot of time trying to 
identify goals and objectives. We search for the right information, 
the right data, and then we collect it and analyze it But v don't 
often enough start with the key questions: What are our values? What is our philosop&y, and 
how do we want to think about it? I honestly believe that leadership and team building first 
require a philosophical understanding and commitment Then you can plan and execute. I 
would like to use a metaphor because we come from such varied backgrounds. I would like 
you to move from being left brain-logical, rational folks- to being right brain-intuitive- 
thinking folks-and participate in an experience that will allow each of us to look for the 
elements of leadership. 



What is leadership? We know good leaders when we see them. We certainly know ineffective 
leaders when we analyze consequences. But what are the conditions that allow leadership to 
occur? How can you get everyone to work together with common goals and objectives? How 
can you participate in the university setting with academic afTaiis, student affairs, 
administrative affairs, and feel that you're all working towards a common objective? Ponder 
the role of institutional research and the orchestration of our professional organizations. How 
does that really work? And most importantly, please don't expect specific facts. 



(At this point in Robert Taylor's speech, the audience viewed the Him, The Bolero," produced 
l>y the Pyramid Film and Video.) 



ERLC 



8 



A Metaphor The Symphony Orchestra 



A symphony orchestra is an interesting metaphor for the organizations in which we work. 
Some of us provide the melody; others provide the harmony. Just as the music evokes 
different emotions in the audience, it creates different emotions for the musicians. We have 
the same responses in academe. What some find exciting, other people do not. And 
sometimes it's very difficult for us to meet around the same table or at the same institution 
aad share a common mission when we bring different approaches to the situation. Achieving 
harmony is something that we really work hard to achieve in the university setting. Did you 
note that there is great harmony in the theme? The musicians come to the symphony with 
very different motivations. They love music. That's why they're there; they just love it. The 
same can be said for those who work at the university. Each person-faculty or staff-has a 
unique expectation about his or her work. 

So when you look at a university, you see people who have come with a variety of backgrounds. 
Often when we speak of the faculty or the staff, we assume that somehow we hold common 
\aliies about why we're here, but in truth, we do not. We hold very different values, and we 
hope that the university setting will provide a place where we can express those values. 
Sharon ? common vision is something we have to work very hard at. And quite frankly, the 
only time this happens is when you develop a plan, in which you specifically articulate the 
mission and objectives. Musicians have involvement and ownership; they really care about 
how they perform their piece of the music. And when you work hard on a project and present 
it to a group of deans and vice presidents who seem to have no understanding of what it 
means. You get frustrated. Your audience only seems interested in picking apart a little 
result here, a statistical analysis there. And you see and you feel your own pride just fading 
away because e^en though you did a good job, it wasn't accepted or reflected in this symphony 
of the university. 

We have a hard time in universities because every one thinks of their own discipline, their own 
instruments, their own section as the most important, and they fail to recognize that to have 
a symphony, you have to have many other people. The leader clearly orchestrates the results. 
And the spirit he or she creates at the end is conveyed by a smile on his or her face that lets 
win know you're doing it right. You know it's right. And that spirit is also exemplified when 
the first violinist (or dean?) starts the v iolin passage with energy, and the others respond with 
gusto. That's enthusiasm. What a difference that spirit makes! Doesn't that" make a 
difference in your own unit and in your own university* 

I am very excited abo.it the opportunities that confront us. I hope you all see what the 
exciting opportunities are for your units. If this is the jear we have to reallocate resources, 
we know it's going to he difficult. We will have to stop doing some things that we'\e done 
before. But by golly, we are going to get together and do the job. We're going to share this 
as a team, and we are all going to participate. Now the cynics among the crowds say, "Oh, 
boy, here goes the faculty senate and the staff senate, and everybody's going to get all upset." 
Hut believe me, if you can jump into something as challenging as reallocation with a spirit and 
enthusiasm that creates the opportunity for everyone to he winners, that spirit will make a 
diHerence. 
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Well, where does the leadership come from? Clearly, it conies from the conductor. It comes 
from the soloist. I think the message about leadership is that the person who is in the 
leadership or headship rob is not necessarily at any given time the leader. We all can have 
an opportunity to demonstrate leadership. And I have seen that leadership from people in 
institutional research and planning, when people were sitting around trying to figure out how 
to attack a problem and one person said, "Well, here are some ideas, let's think about how we 
can approach them in a certain way." All of a sudden, this person becomes the leader, and 
the rest of us follow. Don't assume, because the president is in the room or the provost or 
the vice president, that those persons are exercising leadership. Rather, look for where the 
ideas are coming from, look for where the energy is coming from, and be willing and able to 
exercise leadership on your own. You are not defined jiM by an organizational chart, but by 
your contribution. You ha\e opportunities for leadership. 

I'm always intrigued by watching the support people, the basses and percussion. Institutions 
have rhythms to them, just like the rhythms the percussion and the bass sections provide in 
the symphony. You have to he very careful to contain the tempo; you can't let it get away 
from you. And what I've seen in higher education in the last few years is that institutions 
ha\e let the tempo get away from them. The) capture a new concept-for example, 
internationalization. -and pretty soon it starts running away with all of the resources and all 
of the energies. We forget what it is we were doing before. Or we take on a new market or 
we take on retention, and all of a sudden the only thing that's important is increasing the 
retention rate. We ge' all these statistics, and we forget about the critical question, "What is 
the mission of our institution?" Always keep in mind as you're asked for data, as you're asked 
for information, as you're developing concepts: What is the rhythm of your institution, and 
how can you help maintain it? Because that's how an institution advances, in tempo, not in 
fits and spurts. 

leadership 

Leadership comes from actions and from attitudes that people have about doing the best that 
they possibly can. Leadership comes from each one of us taking pride in what we do. The 
metaphor applies. Our institutions are much akin to a symphony. Both reflect leadership 
and team building. How often do we ha\e to do things we don't want to do? How you take 
such tasks on is the real challenge to leadership. leadership means that you tackle a task 
that you really don't want to do with the same energy and enthusiasm you bring to something 
you're trying out for the first time. 

I isiotu The first element of leadership is liming a vision, being able to see what others cannot 
see: A future state of affairs for your institution, your unit, your community, or your church. 
What separates leaders from other people is that they not only have a vision, but they are 
committed to Miat vision. They ha>e a loyalty to it. They're t.iergued to achieve it. These 
are not daydreams, because daydreams are just fleeting things. Vision means that you see 
something about the future. Your next task is to get everyone else to see the same thing. 

Communication. The way you do that is through communication, leaders are good at 
lomniunicating their visions. You don't tell people, instead you start sharing stories with 
them. You create symbols so that people identify with the organization and unit. A wonderful 
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set of ads that appeared about a year and half ago in national papers and magazines showed 
employees from the General Klectnc Company with T-shirts that said, "GE is me." The 
symbolism of those T-shirts was very profound, because to the workers that slogan said, "Hey, 
we count in this company." GE had heen fragmented for years hy management-labor disputes. 
To the management people, the T-shirts signified that the employees really did count and were 
to be taken seriously. To customers those ads said, "Hey, those are Americans making our 
products, and they're real people just like me. Pm going to huy from them." Uaders create 
symbols, an image of what it is tliejVe trying to achieve. And that's the special thing they 
bring to communications. 

The most important skill leaders have in the equation of our organizations, however, is the 
ability to ask questions. Another critical skill leaders have is listening People who chair a 
meeting often have all of the answers; they don t need anybody else, they could just do it 
themselves, leaders ask: "Why are we doing this? Is there another way? Do we have a 
better alternative? What do you think?" People at the top of organizations are the ones least 
capable of making the hest decisions because they have the least amount of information. 
What institutional researchers need to do in this leadership dilemma is to help leaders 
understand that the information is to be found where the faculty n.cet the students or where 
the salesperson meets the customer. That's where the primary information is. How satisfied 
students are with the quality of their education can not be counted hy the number of student 
credit hours generated nor by a random sampl » of fifteen students. Rather, we need 
information from the constant daily interaction of the faculty member and the students with 
the institution itself. But that kind .>f information is rarely at the top when decisions are 
made about where we're going to go. How can you help the people at the top to understand 
the value and the quality of information they have, and how can you develop your research 
to provide information for the people who are where the action is, those who desperately need 
it to make the ongoing day-to-day decisions? That's a real challenge for the future of 
institutional research. 

Finally, you communicate a vision through emotions. Because words are just not enough, you 
do it through emotion, which then translates into action. If vour institution is truly 
committed to affirmative action, then everybody involved in making hiring decisions must 
demonstrate that commitment. You don't just put an edict out and say we're committed. 
From the top to the bottom and all of the sides, there must be clear evidence that that's what 
our commitment is. Well the same goes for your institutional mission and for your goals and 
objectives. If you are not accomplishing them in a day o-day way, then they're not worth the 
paper they're written on. 

Taking risks. The third element of leadership is taking risks. We can no longer be 
conservative in academe. We have to take risks. We have to put venture funds into trying 
the unknown. We have to explore new ways of educating people. But our institutions are 
designed to minimise and eliminate risk. And that's a difficult challenge. There are people 
willing to explore and take risks. And I think we need ideas from people in institutional 
research about the kinds of risks we ought to be taking. 

Action. The fourth element of leadership is understanding the process of motivation. You 
cannot motivate anyone; people motivate themselves. You am change people's behaviors: yes, 
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you can do that. You can offer people promotions, you can give them merit raises, you can 
punish them, you can gi\e performance appraisals. But they will decide-the employees, the 
people you work with-whether or not they are enthusiastic about what they're doing. We 
work in bureaucracies in which it's just assumed that a 7.5 percent pay raise will motivate 
everyone to work hard next year. Or we assume that if we give Person A a 10 percent merit 
raise and Person B a 5 percent, somehow we're reinforcing Person A and we're going to 
motivate Person B to try harder. Neither one of those is true. Persons A and B may not be 
there for the money; they may be there for some other reason. They came to the symphony 
with a different set of ohjectives and values in mind. We have to work harder to identify why 
people at universities work there and what will keep them active, enthusiastic, and involved. 
Simply juggling an annual salary allocation is a very shortsighted way of approaching this 
issue. 

Values. The last element really has two parts. leadership is loyalty and commitment, but it 
goes both wa>s. loyalty goes up the organization and, more importantly, loyalty comes down 
through the organization. If we're going to encourage risks, we don't slap people's wrists when 
they n i ;ike a mistake. Instead we encourage people to make mistakes and we encourage 
conflict because that's how change takes place. I^arge bureaucratic organizations tend to want 
to minimize mistakes. Afraid that we'll look bad, we try to cover up, move aside, or just 
reformulate the results so it doesn't look like we made a mistake. Yet my study of successful 
leaders reveals that, at one time or another, they made a serious mistake. What they learned 
from that experience contributed to their success as a leader. If we in administration want 
people to have loyalty to our institutions, then we'd better have loyalty to people who work 
with us. One of the fascinating things about President Reagan was his ability to inspire 
incredible loyalty from the people in his cabinet and the people who worked in his 
iiclministration. When those people got into trouble, he remained loyal to them even when 
public opinion went otherwise. 

The most important \ahie that leadership requires is trust-a rare commodity in academic 
institutions. You cannot ha\e leadership without trust. You can have dictatorship, you can 
have headship, you can have management, you can have administration; but leadership comes 
onl> when people start to trust each other because they share common values and they share 
a common goal for the institution. 

Team Building 

What have we learned about team building so far? Well, one thing we learned is that you 
have to be looking at the conductor. You have to be honed in on the president while you are 
listening to what's going on. (let out of your institutional research ofTices and walk around 
the campus, Find out what's going on. What are people's concerns? Talk to the students; 
don't talk to deans because they don't know what students are thinking about. When you talk 
to students, get a sense of what your data mean relative to what you're studying. I would love 
to have that occur. Then, talk to people who aren't students (and should be) and find out 
win. That's what we really need to think about, 

We get so set on reading our journals and professional literature about key issues. Quite 
frankly, I'd be much happier if institutional researchers were reading existentialism, listening 
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to country music, a.id just going out and cruising the campus every once in a while. Then 
read the professional journals. I think you learn more in our changing environment because 
our professional literature merely has us extrapolate the past. The papers were researched 
two years ago and the data were collected two years before that. Is that state of the art? Our 
environment is changing so fist thut we can't afford to worry about what happened three years 
ago. We need to be tuned into the future. I learn mere from popular music lyrics about what 
people are thinking than I do from any research paper I've read. 

The rmphasis has to be on the grou|~ and not the individual; whether we're talking about your 
institutional research group or the whole university. I see universities in which one school 
or one unit is the stellar one, and everybody just kind 01 centers on that-the pride and joy. 
The rest of the university stagnates. 

I see institutions in which the individual is so important that people forget the fact that you 
have to have everyone there. Sure, all those soloists are good, but that doesn't mean anything 
unless they can play together. People need to learn that whether they are administrators, 
faculty, or staff, they can be soloists every once in a while, but they also play in the chorus, 
and both are equally important. Sometimes soloists are so excited about their solo that that's 
all they want to do. 

Feedback and evaluation are important. But we must look at the way feedback is given. The 
idea of getting feedback from people is not having the answer and saying, 'That's not right, 
that's too much, that's not good enough, I want to see better out of that unit," but rather 
asking the questions. You can help formulate the right questions. 

One thing I like to do is meet with the folks in our institutional research area every once in 
a while and ask for their perspective of what we're doing in the business school. 1: stitutional 
researchers an? marvelous at asking questions ihat get me to really think about what we're 
doing and how we're doing it. This is much more valuable than trying to do it myself because 
I'm biased hy what I do every day. That's an opportunity you offer to your campuses. But 
you've sot to go out and share that feedback Feedback i* only important in a team if it's 
open and accurate. 

Changing Environments 

Institutional research is changing. The variables that you need to be concerned with are 
shifting dramatically because the world about us is engaged in so much dynamic evolution 
that the data that have been collected for the last thirty years are not the data we need for 
the next ten. We don't need market research on who's going to buy our brands, but on what 
the opportunities are to provide education for our constituencies. 

My hypothesis is that urban schools will spend less of their time educating undergraduate and 
graduate students in the next twenty years and much more time providing continuing and 
professional adult education. That will be the source of most revenues twenty years from now. 
There are not man> that share this belief. What will the research reveal? 
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rurther, we shouldn't he worried about the number of 18- to 22-year-olds but we should be 
very concerned about the higher education illiteracy that we have in c ir workforce. Because 
work is changing dramatical!), employees need to be educ:**' J and they need to upgrade their 
skills. Universities are poised to do that, and those people range from age 18 to 68. We must 
reassess what our markets are. We're still worried about the number of 18- to 22-year-olds 
while industry spends more on education than the combined higher education budgets 
throughout this country. We should be in that game. 

Universities must build on strengths. But oftentimes when you have a group of soloists 
screaming that the) are the most important, you need an objective evaluation. We need some 
input as to which are the strengths: What are the most important sections, where are our 
best soloists, where could we have fewer violins and perhaps have the same rich sound in the 
orchestra? You see, an orchestra can i.e 110 pieces, it can be 55 pieces, it can be 70 pieces, 
it can he all sizes. For years, we've bren saying that each one of us is going to be an 
orchestra. Some of us started out as en»emhles; some of us started out as string quartets; 
some of us started out as orchestras; ard I think we all believe we should be all 110- piece 
orchestras, able to play any piece. 

The demographics are changing, yet everybody is still trying to be an orchestra, and players 
ai e leaving. Subscribers aren't coming to the concerts. And all of a sudden you're struggling 
to find out what you are. It's better to be the Juliiard String Quartet than an orchestra with 
34 pieces, all of them bassoons. 

Yes, the < :stra has limited resources. The conductor carefully schedules rehearsals. The 
time limitations are very strict. That's very much what we're faced with, The score, the 
outcomes assessment, is the biggest single opportunity and challenge you face in the next five 
years. Quit counting inputs and start helping us count outputs. Because if you don't, those 
of you in public institutions will find that state legislatures are going to do it for you. And 
those of you in private institutions are going to find that when a few institutions siart 
advertising that they can measure the quality of their students' $15,000-a-year investment, 
you'll be asked wh) your school can't. And believe me, parents will have their children vote 
with their feet. We need meaningful outcomes assessment. 

The Role of Institutional Research 

Well, what role will you play? Aie )ou staff advising the president? Or are you the first 
violinist, trying to get a team to make sure they're all playing the melody with the same gusto, 
"bo\»*ng" together? Or is planning what you do--do you write the score-that others must 
execute? 

1 see institutional research as one of the most strategic roles of the twenty-first century. 
Because as we move into the information society, if you are able to collect the right kinds of 
information, you will be the most powerful part of the university setting. You must be able 
to identif) what that information is, to bring it in, to analyze it, and then to disseminate it. 

In American industry for the last twenty years, whoever assembled the most financial 
resources was the most powerful. Those organizations are disappearing. They're being 
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replaced by those service industries that can identify where the markets are, identify where 
the customers are, bring the data together, analyze it, and make a match. Those are the 
organizations we must clone. 

Information will be power. Because you're in the business of identifying and acquiring 
information, analyzing and disseminating it, you are the key pivotal unit I see this as a detn; 
I see thix in relation to industry; and I see this in terms of the orchestra. Planning and 
research are the key instruments in the ensemble of the university. The conductor, the score, 
and all of us as players must work together in bringing our visions for our institutions into 
reality. 
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What Don the r ;turc Hold for lnstitutiosuri Research? 

(general session) 




George Keller 

Senior Fellow, Graduate School of Education 
University of Pennsylvania 

(This is an edited record of an address presented in the Grand Ballroom at the Gait House 
in Louisville, Kentucky, on Monday morning, May 14, 1990.) 

When AIR's forum chair, Dr. Edward Delaney, asked me to speak 
nearly a year ago, I was reluctant because I am not now and have 
never been an institutional researcher. But Ed spoke in his usual 
soothing tones and said, Don't worry, we'll send you all the 
materials you need to bring you up to speed.** I had no idea how 
thorough he would be. He enlisted Dennis Hengstler and Jean 
Chulak and rubers to send me materials. Before long, my mailbox 
was full of books, articles, and pamphlets~everythlng from Muffo 
and McLaughlin's A Primer on Instautumal Research to several 
volumes of New Directions for Institutional Research and dosens of 
papers from your AIR Professional File. I was also encouraged to 
study issues of Research iu Higher Education and to 'oterview my 
friends in institutional research, which I conscientiously tried to do. 

1 now have a sense of what it's like to have the National Rifle Association or the American 
Association of Retired Persons lean on you. I have no idea how you people keep your sanity. 
1 read about student flow models, economic impact studies, the help you provide with 
institutional self-studies, program evaluations, the work you do with the budget, retention 
studies, data for strategic planning, data for fund-raising such as donor research and alumni 
records, and reports to the federal government, the state government, and state higher 
education commissions. I read about environmental scanning, program needs assessment, 
advice on the purchase of hardware, software, and networking, competitor analyses for the 
admissions office, statewide, national, and international comparisons. The portfolio is 
breathtaking. The number of tasks you people do would push a normal person out of sanity. 
So I can only surmise that most of you in the audience are abnormal 

I was helped in my understanding by a wonderful article that Laura Saunders wrote ten years 
ago, in which she said, "The institutional research analyst, as a result of the special work of 
1R, must work under great time pressure, must contend with a large volume of pressing 
concerns, each competing foi attention and must have access to and be knowledgeable about 
sources of information within and without the institution, and the analyst must do this all 
rapidly, accurately, and with good humor." I have a sense that I am speaking not to the 
Association for Institutional Research, but to the Clakk Kent Society. 
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An Art, Not a Science 



I've been asked to speak about the future of institutional research. As you know, looking at 
the future is a hazardous assignment. The old saying goes, "He who lives by the crystal ball 
lives to eat broken glass." Nonetheless, I will talk about a few of the changes I see for y< u. 
I would like particularly to call your attention to two major developments that I believe will 
shape your future. 

Before I talk about these two major developments, however, I'd like to express an opinion 
about the continuing argument about whether institutional research is or can be a science, 
or whether it is and always will be an art. Yon will hear more eloquent and methodical 
comments on this subject tomorrow by our fellow scholar from the Netherlands, Frans van 
Vught. I just want to offer a few remarks. 

I believe th.it institutional research will continue to be an art or nigh-level craft demanding 
great skill and meticulous craftsmanship. It will continue to be more a profession like 
architecture than a science like molecular hiology. For one thing, many of the finest people 
in the social sciences are coming to helieve that the social world may not be as susceptible to 
scientific lawfulness as many people thought as recently as a decade ago. Skepticism about 
a science for social behavior is growing like a mold over all social studies, even in the field 
of economics. For another, I think weVe beginning to r*a!ize that science, which has great 
power in the material world, has hecome an ideology in social studies. 

We have come to believe that science is somehow a higher activity than statesmanship or 
craftsman-Iike behavior. Yet, I^onardo da Vinci was not a lesser person than Galileo; nor is 
interpretation any less intellectual than mathematical logic. As Ludwig Wittgenstein said, 
The existence of experimental method makes us think we have the means of solving problems 
which trouble us though the problems in methods pass us by." Anthropologists such as 
Clifford Geertz and others are learning that being emotionally close to the subject can often 
yield more understanding and accuracy than scientific ohjectivity. William James, a great 
American psychologist, observed that loving is a deeper form of knowing than detachment. 
And many of you, I know, love your institutions and the world of higher learning. 

For better or worse, institutional researchers seem to be married to institutions of higher 
education. Intimacy is a help, not an impediment. IR people will always need to know their 
organizations and the world they live in as Shakespeare knew humankind, and not as 
scientists know the motion of planets or the tiniest particles. 

Twin Pulls: Centralization and Decentralization 

The first of the two major developments that will affect the future of institutional research 
is that of the twin pulls toward greater centralization and greater decentralization of 
information in higher education. This is not a new tension. Marv Peterson has written 
beautifi lly about this, Peter Fwell has talked about it. Kdward Delaney has written a nice 
article, and numerous others of you hate seen this coining. 
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Centralization is required by the growing demands for accountability, for accurate reports, for 
strategic and financial planning, for quality control across all the schools and departments, 
and for comparisons within institutions and with peer institutions. COFIIE, NCHEMS, and 
numerous others have encouraged standardization of data on the campuses so that 
accountability can he swift and comparisons useful. These demands require a central 
information bank and probably a chief information officer, or CIO, such as numerous 
unhersities have begun to install. There is a powerful pull to collect all the disparate 
information at institutions and at some central and easily accessible source. 

On the other hand, the forces pulling toward decentralization appear to be equally powerful. 
The first is the need to cut costs. Just as the 1970s and early 1980s were a period of political 
controversy, when we worked to equalize opportunities for females, for minorities, and for 
people of different sexual preferences, so the 1990s will be 8 period of financial controversy. 

Education costs in the United States, Canada, and most other countries are rising about 20 
percent faster than the consumer price index. Education is a handicraft, primitive industry 
that is labor intensive, and it uses high-cost labor at that. It is difficult to make ourselves 
more efficient, to run our institutions more cheaply. Students demand personal contact with 
first-rate intellects and it is hard to use the benefits of technology for teaching. As a result, 
we are a segment of the economy like the arts, like great restaurants, and a few others, where 
e\erything is still done in artful and handicraft fashion at great expense 

What this means is that to keer costs reasonably within control, educational institutions are 
increasingly establishing responsibility centers on their campus, giving deans their own 
budgets, and giving various departments and offices their own budgets. Administrators are 
asking deans to hecome more entrepreneurial, to raise money for their own enterprises. While 
reading some papers for the Association for the Study of Higher Education, I noticed a paper 
from Texas A&M that explained the achievements of something they call the senior academic 
business administrator, a special position in each of the schools of Texas A&M. In other 
words, there is a financial-and-academic-information collecting-officer assigned to each of the 
deans. Texas A&M, like other institutions, is decentralizing the collection of economic, 
academic, and other kinds of data to make them immediate!; available to the institutional 
units. 

\t my school, the Graduate School of Education at Penn, our associate dean, Dr. Michael 
lierne), who is a wizard with computers, nas increasingly taken on the role of our chief 
information officer. He devises budgets and collects data from other parts of the university 
to compare us with other schools within the university, and with other graduate schools of 
education around the United States. More and more of the units within each university will 
be increasingly driven hy the need to pare costs to keep themselves efficient, and they will 
need to develop institutional research to accomplish this. 

Another factor driving decentralization is the new technology, which all of you know about: 
the micros, the graphics, the profusion of software, and all the rest. I'm not alone in thinking 
that this will also decentralize. Peter Kwell wrote in New Directions for Institutional Research, 
Number 64, "'I he most important advances in micro-computers and in electronic 
iomiuunicatiou promise to alter the traditional institutional balance of power with respect to 
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information in fundamental ways. No longer the sole supplier of decision information, 
institutional research is increasingly becoming only one of a number of independent players/' 



The advances in technology will almost inevitably give rise to an information feudalism, with 
semi-independent computer domains all over the campus. This could result in a kind of 
anarchy of information on most campuses, since sophisticated individual computer users can 
get into these data sets, and with increasing ease do their own studies in areas that interest 
them. Networking will increase this possibility. Add to this the faculty's independence and 
their increasing ability tc gather data and assemble it for their own personal and academic 
uses. 



These twin pulls toward centralization and decentralization will in the years ahead tear 
institutional research in half. It has already begun to do so. One option is to tilt toward 
becoming a chief information officer-the broker of all the feudal domains on your campus, 
a policymaker on information, a coordinator, and a setter of information standards and 
categories for other information collectors. Or you can form a partnership with the vice 
presidents or deans or other unit heads to help them with their particular data needs with 
their budget projections, with their attempts to trim dollars from their budgets, and to 
innovate arademically. 

This is not an easy decision to make. The second option, unlike the CIO line role, is basically 
a staff role. Instead of being in the president's administration building, you will be with the 
local unit heads. It will require different loyalties. It will require different outlooks. The CIO 
will be increasingly responsible for keeping all the data networks reasonably honest, while the 
unit IK people will be helping individual deans sharpen the data and then bend it a little for 
their partisan management needs. The two roles will require different operating styles, 
different attitudes. 

External Change versus Internal Demands 

The second major development is also a twin pull with a potential for tearing institutional 
research in half. Institutional researchers will increasingly n*»ed to monitor and document 
change and novelties, especially in the outside environment. Yet they must also respond to 
the here and now-the internal demands of the next budget, management information system 
requirements, surveys and reports to agencies. 

The need to document change, scan the future, and collect data from outside the campus 
arises from the fact that since the mid-1970s, the United States has been going through a 
most traumatic change. The position of the United States has been changing in all sorts of 
ways, and any line of data you pursue will have sharp breaks in it somewhere between 1973 
"hen the OPEC oil cartel was formed, and 1976. U.S. society will be forced to adapt to the 
new conditions. 

The conditions are staggering to contemplate. We are changing demographically as a nation 
with birth rates below replacement, the highest immigration smce 1910, and a new geriatric 
society. By the year 2020, according to the U.S. Census Bureau, approximately 19 percent of 
our population will be over 65, and 9.8 percent will be over 80. Imagine a society that is one- 
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fifth elderb and one-tenth \en old. <Belie\e it or not, the fastest growing group of enrollees 
in higher education is the o\er-55 group.) Colleges and universities are becoming less and 
less training camps for the young and more and more like public libraries, open to teenagers 
during the summer, and to traditional students, adults, and the elderly. 

Socioeconomic changes in this countn are equally profound. The American family as we 
know it is disintegrating before our eyes. The family that I grew up with a father working, 
a mother at home, and two or more children now constitutes less than 7 percent of America's 
families. A fourth of all babies in the United Stales are now born out of wedlock, compared 
to 4 percent for Japan. And even when children live with two parents, 66 percent of the 
women are working outside the home. Ibis has profound implications and is already affecting 
our schools. In years to come, it will affect our colleges and universities. Young people lack 
adult guidance, motivation, and discipline, and this influences student behavior and student 
aspirations. The effects are already apparent, particularly in the black community. 

On the economic scene, the United States has slipped from being the richest country in the 
world to fourth, fifth, sixth, or seventh, depending on the year. We have lost about 20 million 
manufacturing jobs to South Korea, Hong Kong, Singapore, Brazil, Italy, Portugal, and other 
nations. There is no longer a single radio or video cassette recorder made in the United 
States, nor a single baseball or pair of jogging shoes. The economic change the United States 
is going through will affect e\ery segment of higher education in a general way through the 
tax base and in a specific way for academic schools such as agriculture and social work. 

In the cultural area, the arts !ia\e become more important to many youny people. The 
United States is becoming one of the most culturally active nations in the world. The data 
from L>nne Cheney's National Kndowment for the Humanities shows that more people go to 
museums and art shows each year than go to athletic contests. 

In religion, more and more young people are asking what their church can do for them rather 
than subscribing to whatever ihe priests and rabbis and ministers urge them to do. In a 
strange way, religion has become a consumer activity, a development that has affected every 
religiously -oriented college m this countn. 

I "on't talk about the technological resolution we are living through. You know that 
intimately. But the fact is, the computer and all the telecommunications that can hook up to 
it constitute the most important change to affect education since Gutenberg's invention of the 
printing press. The implications are profound and weVe only beginning to sense the degree 
of networking that can be done among colleges and universities. 

I lien there are the political changes. We have lived for a long time under the threat of 
communist imperialism from the Soviet Union, but this has suddenly collapsed. The Soviet 
Union is actually in danger of being dismantled somewhat the wav the British empire was 
dismantled after World War II. 

Colleges and universities, some to survive, most to maintain quality and service, and all to 
adapt, wilt need better information about the nature, scope, and significance of these 
extraordinary changes taking place outs.de the campus gates. The new conditions in the 
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United States will pull institutional researchers increasingly towards new forms cf 
reconnaissance about the fit between the en\ironuient and colleges and universities. Those 
colleges that have the best information from people like you, that have the keenest alerting 
mechanism, will be able to innovate intelligently, seize new academic opportunities, capture 
foundations and government grants, and impress with their public spiritedness in ways that 
colleges that do not carry out first-rate reconnaissance on the external environment will not. 
Though Jim Morrison of the University of North Carolina, Rick Heydinger of the University 
of Minnesota, and a few others have been pointing the way, we still know relatively little about 
how to gather trend data, how to do sensible forecasts, and how to provide information about 
the rapid shifts going on all around us. 

This external, future-oriented focus will be balanced by the other pull: the need to provide 
current data about the internal operation. This day-by-day information will continue to be 
indispensable. It is as vital xo monitor what is going on right now inside your own universities 
as it is to scan the outside environment to find out what might happen to your universities 
in the future. 

But the two processes for gathering information are quite different. One gathers data heavily 
from the outside to affect hternal changes. The other collects data mainly about inside 
activities largely for external monitoring bodies such as state agencies or the federal 
government. It will be difficult to serve both these masters. This conflict could also tear 
institutional research in half. 

What do these changes portend for institutional research in the 1990s? Well, I can imagine 
a range of possibilities. One is that the Association for Institutional Research will remain 
pretty much as it is now, a marvelous and polyglot body of folks who have something to do 
with information in many parts of our institutions. I can also imagine, Vm sorry to say, the 
near dissolution of institutional research as we now know it, with the work of what is now 
regarded as institutional research being broken up into many sections and being attached to 
offices of planning and finance, the offices of the deans, the president's office, the physical 
facilities office, and many others. 

What makes this difficult to predict is that onh part of your future is in your own hands and 
mine. The campus executives, the leaders at your university, play a large role in deciding 
what kind of information their institution should have, and these campus executives have a 
rapid turnover. Campus executives at small Roman Catholic or Baptist colleges want 
different kinds of information than those at elite universities or at two-year technical 
institutions. It will not be easy for >ou to shape your own future given the position that AIR 
people occupy in university life. 

Tiro Observations 

I would like, in conclusion, to make a couple of observations. One is that if you worry about 
the future of institutional research, I hope you will keep in mind that higher education is 
redefining almost monthlj what it needs to manage itself. University administration in this 
country, and I presume in other highly industrialized countries, is changing right before our 
eyes. We have new kinds of administrators. We have new forms of governing. The budget 
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and priorities committee, Tor instance, or the strategic planning group, is increasingly being 
seen as an advisor}' cabinet government within universities. We have seen the rise of 
computer czars, of chief information officers. At the University of Pennsylvania, we recently 
hired Dan Updegrove from EDUCOM to look at our entire range of computing and 
informational activities at the University of Pennsylvania. Other institutions already have 
similar people. There are new vice presidents for enrollment management. We have suddenly 
realized that the admi ssions officers at ever}' private institution in the United States are the 
chief fund-raisers, not the development people. Admissions persons bring in 50 to 7C percent 
of the income at most private colleges and universities, so we have stopped treating 
admissions officers and financial aid officers as clerks and have begun asking them to be 
executives like the pro\ost or the financial officer. We have new affirmative action vice 
presidents. We have an increasing number of chiefs of assessment and evaluation on campus. 
We ha\e assistants to the president for planning. In our finances we are moving from the 
world of fund accounting to a whole new kind of activity--what business firms call finance. 
The entire academic and administrative portfolios are in flux. So if institutional research is 
in flux \ou need to realise that \irtuully every other office on campus is also changing. 

M> second concluding observation is that we usiiaPy have more influence than we think, and 
more power to help shape our futures than we think. Too many of us, it seems to me, have 
become prisoners i>i social studies that emphasize the powerful social forces that shape our 
Ines. Ihe social sciences stress determinist causation and harp on the feebleness of 
leadership. And our humanities have increasingly neglected biography and similar studies 
that emphasize the role of the individual. But I hope that Eastern Europe makes us aware 
of the potential for change from ordi .rv individuals. Colleges and universities v be altered 
b> institutional research officers, by first-rate provosts, extraordinary deans, or by a cadre of 
faculty who really care. 

It is important that you see AIR as a force for influence and change and not as a victim of 
change. How can you do this? What should AIR be doing in the years ahead? It's 
presumptuous of me to suggest, but let me throw out a few things that you might consider. 

What Should AIR Do? 

One is to develop a kind of scholarship of your own. With all the power of the thousand 
people in this room, and with your colleagues in SCUP and CAUSE, and with the professors 
in the universities that yon work at. you should be able to develop an agenda of likely 
informational needs for the next decade. Seize the initiative and tell colleges and universities 
what the information needs of the coming decade will be, rather than waiting for your 
presidents, your deans, and >oiir board chairpersons to tell you what data to gather* The time 
has come for this organization to move from a passive and laissez-faire attitude to one of 
leadership in the field of information. I think there are foundations and corporations that 
would support such a major scholarly effort. 

Secondly it is important that you stay close to and develop ties with neighboring 
organizations in higher education such as SCUP, CAUSE, and ASHE. These organizations 
cpreseiit the scholars of higher education, the computer experts, and the planners in higher 
education. I think there will he all sorts of points at which you will be able to collaborate. 
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Last, I suggest AIR And some way to train the institutional research people of the future. It 
is one of the strange anomalies of this profession that you people have become powerful 
arbiters of policy and vital providers of information on campus, yet you come from the most 
diverse backgrounds and scar ly any of you have had any training for the work that you do. 
As most of you know, the definitions of a profession are a first-rate training program, a body 
of knowledge that can be mastered, some sort of test or license such as the bar examination 
or CPA exam to see if one has mastered the body of knowledge, and a self-policing mechanism 
to chastise the culprits among you. Unless AIR begins to develop a body of knowledge of what 
it takes to become an IR person, you will not have the opportunity to establish yourselves as 
scholarly professionals on your campus. 

If it is true that we live in a qualitatively different kind of information society, and if 
brainpower and the colleges and universities that provide brainpower are the keys to the 
future, then it seems inevitable that information officers at colleges and universities will have 
a central and perhaps powerful role to play in the coming decade-not just in higher education 
but in shaping the civilization we live in. 

So on the one hand, you face n tearing asunder of your profession, a feudalization of your 
profession. On the other hand, the work you do has never been more important if this society 
and other industrial societies are to remain strong and free. 

I counsel you to be brave, to think boldly and innovatively, and to remember that the future 
isn't what it used to be. Recall again the words of your former president, Laura Saunders, 
The analyst must do all of this rapidly, accurately, and with good humor." I would 
underscore the good humor. 
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A Personal Retrospective on the Development of Iastftutkmal Research 

(general session) 




C Robert Pace 

Emeritus Professor 

University of California at Los Angeles 

(This is an edited record of an address presented in the Grand Ballroom of the Gait House 
in Louisville, Kentucky, on Tuesday morning, May 15, 1990.) 

The theme of this session, "Back to the Future,** suggests that 
somebody should lock back. My own history includes some 
encounters with what is now called institutional research, 
beginning nearly sixty years ago. I would like to recall some of 
those encounters and suggest what we all might learn from them, 
or at least what I think I learned from them at the time. 

In the 1930s, when I was working on my doctor's degree in 
psychology at the University of '/finnesota, I had a job as a 
research assistant with the Committee on Educational Research. 
One day the president of the university said he wanted the 
committee to survey the economic and occupational status of those 
who had graduated from the ins titution during the Depression years. So we sent a 
questionnaire to about 14,000 students who had graduated between 1928 and 1936. What we 
learned from that survey is important for us to remember today. A baccalaureate degree is 
not an insurance policy against the effects of an economic depression. 

My first major encounter with institutional research was during the three years I was 
employed by the General College at Minnesota to direct an extensive questionnaire survey of 
former Minnesota students. The General College was a new enterprise. It targeted a group 
of students that hitherto had not received specific attention from any college of the university, 
•mainly students from the lower half of their high school graduating class. The General 
College curriculum addressed the present and future needs of these students with respect to 
vocational activities, home and family life, social-civic affairs, and personal life. All of its 
courses dealt with contemporary aspects of life. The central purpose of the survey was to help 
faculty members decide what to teach. What did the faculty want to know about the activities 
of young adults, their interests, the problems they faced, what they wanted to know, their 
attitudes and opinions? Have you ever heard of using a survey of former students to determine 
the curriculum and the content of courses? It seemed like a good idea. The results were directly 
reported to faculty members who said they wanted the information. 

The other aspect of this survey-which I think may be an all-time record-is that the 52-page 
questionnaire was returned by nearly 70 percent of those who received it On average it took 
most people an hour to an hour and a half to respond to the more than 1000 items. Why did 
they do it? The questionnaire was very attractively printed and illustrated; its content was 
obviously relevant to the lives of the young adults who received it; it came from a high 
prestige sotuce; and it haH a commendable purpose. People who received it said such things 
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as: "I showed it to six other persons '; "My husband and I found it interesting and read it 
from end to end": "It took a long time to fill out, but it was a pleasure." In fact, nearly 100 
of the more than 900 respondents asked if they could have a copy to keep. We used a good 
cover letter, two follow-up postcards, then a two-page letter and two more follow-up postcards. 
We did everything right, and the result was very good. I'm sure the survey's success was properly 
attributable to the extensive planning and careful execution of the whole enterprise. 

Many years later, at a time when most people considered themselves lucky to get a 50 percent 
response to a much shorter questionnaire, it occurred to me that the reason we got such a 
fantastic response to such a long questionnaire might have been that we sent it before the 
invention of television I 

In 1947 I joined the faculty of Syracuse University. For the first five years my main job was 
in Uie Evaluation Services Center, and during four of those five years, I was also a special 
assistant to the chancellor. Syracuse was just embarking on an extensive self-survey and the 
Valuation Center was to provide technical and coordinating services for all of the survey 
committees. Maurice Troyer, director of the Kvaluation Center, directed the self-survey 
project and I served as assistant director. The trustees had asked the chancellor for a survey 
of material needs that would be useful in projecting a financial plan for the next five years. 
The chancellor pointed out that no financial plan could be projected without also first 
assaying the quality and quantity of the educational program and services-curriculum, staff, 
library personnel, administrative organization, plant and facilities, publications and public 
relations. In projecting needs, the university would also have todetermine the size and nature 
of the student body it planned to serve. 

During the 1947-48 academic year, about one hundred faculty members and administrators 
were actively engaged in the self-survey. At the end of the academic year, each survey 
committee prepared a detailed report of its activities, its evidence, and its recommendations, 
and each report was reviewed by the chairmen of all the other committees. My recollection 
is that nobody got any time off for the hundreds of hours put in.o this self-suivey. 

The committee reports, in total, ran to more than 1000 pages. I was asked to prepare a 
summary report of the survey that would be circulated among all the committee members for 
information and comment. In the fall of 1948, representatives of the survey committees met 
with the university's top administration to discuss the findings and develop plans for 
translating recommendations into action. This was a two- or three-day meeting at a university 
lodge in the Adirondacks. Basically, the faculty members said to the chancellor, the vice 
chancellor, and the business manager, "Here are the survey recommendations; what are you 
going to do about them?" 

What happened next at that meeting in the Adirondacks was a very interesting and sharp 
difference of opinion about what to do with the suney reports. The group agreed that the 
detailed committee reports would be on file in the Chancellor's Office and in the Evaluation 
Center, and that authorized personnel could see them. As to any wider circulation, some 
people argued that the material was too sensitive. The Syracuse newspapers would have a 
field day if the* got hold of the reports. Others argued that the benefits of the survey would 
be lost if the results were not shown to the entire faculty Some suggestions were made for 
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modifying t» few paragraphs of my summary report, and I was asked to prepare a revised and 
s< mewhat shorter version. 

The chancellor decided to issue a report to the faculty. We got galley proofs of the text and 
they sat on the chancellor's desk for a couple of weeks. He made a few additional changes, 
but then he left town for a month. So I took the galley proofs off his desk and sent them to 
the printer. When the report was printed, I decided to mail a copy to some of my friends who 
might be interested-Ralph Tyler, Earl McGrath, Al Eurich, T.R. McConnell, Ruth Eckert, and 
;i few others. Not long afterward, while the chancellor was still out of town, several people 
approached him at a conference to tell him, "Bill, I just read the report of your self-survey and 
I want to congratulate you and the university. Any university that can make such a forthright 
appraisal of itself must lurte good faculty morale and confident in the future." At that time, 
of course, the chancellor did not know the report had been published! Anyway, the result was 
that the report was very widely circulated among the faculty, and for the next few years, 
several departments sent copies of it to persons they were hoping to recruit. 

1 do not advocate the directors of institutional research take the sort of risk I took! However, 
I do think it is very important to understand that, in institutional research, there are no secrets* 
That is especial I) true toda) with computers and act.ss to whatever is stored in them. 

Three >ears after the self-sun e\. the chancellor asked if we knew what had been done about 
all the survey recommendations. How many had we acted on? Could we do a follow-up? We 
could and we did, but I persuaded him that if he was willing to wait a few months for an 
answer, we could use this opportunity to analyze as well as count whatever actions had been 
taken. One of the research assistants in the Evaluation Center had been thinking about 
possible dissertation topics. A follow-up study of the self-survey recommendations seemed like 
an excellent topic to him; and indeed it was, for Arthur Browne wrote a very thorough and 
creative dissertation. From the complete reports of each committee, he identified some 500 
recommendations as calls for action of some sort. He then classified each recommendation 
along six dimensions: (1) the area or topic, (2) the type of action called for, (3) how much 
action had been taken, (4) the probable cost of the action, (5) the intended beneficiary of any 
action, and <6> the le\el of responsibility required for any action or decision. By cross- 
tabulating the results of these classifications, Browne was able to answer such questions as: 
What happened to recommendations related to curriculum and instniction that could be 
implemented at the faculty or departmental le\el? What happened to recommendations for 
providing a facility, involving major cost that would have to be authorized by the Board of 
1 rustees? Was there a predominance of recommendations for which the faculty itself was the 
intended beneficiary? In short, the follow-up study analyzed the dynamics of action as well as 
the impact of the self-sune\ on changes at the university. J tell this story to illustrate the value 
of having institutional research connected to graduate programs. It can enrich the quality of analysis 
and enable students to learn something important ahout higher education. 

\ few >ears after the self-sun e> was completed, the university was due for an accreditation 
re\iew by the Middle States Association. Since we had already done a very thorough self- 
stud). \er> little new information was needed. One Thursday afternoon at about four, I 
iecei\ed a phone call from the wee chancellor. He said that most of what the Middle States 

\ssociation wanted had been assembled, but could I prepare a response to two very broad 
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questions? He did not want a detailed answer, just an overall response of a few pages. The 
questions were: What are the objectives of Syracuse University? and What are the directions 
and changes the university hopes to take in the next ten years? He said there would be a 
meeting of administrators in K office at eight thirty Monday morning, and could I please 
have something to report at that time. I arrived with my response on yellow scratch paper, 
and since no one could possibly read my handwriting, I read what I had written to the group. 
When I finished there was no Immediate response, just silence. Finally, one of the 
administrators said, "I can't think of anything to add or subtract from what you have read." 
We had the statement typed and circulated to other people for comments and revisions. What 
subsequently appeared in the report to the Middle States Association was about 95 percent 
verbatim of what I had written. I do not tell this story to suggest the value of having majored 
in English composition in college. I tell the story for a much more important reason. If the 
office of institutional research is indeed a center of information about an institution, then any 
director of the office who has been on the job for at least a few years should be able to respond to 
the two questions I was asked. Tlte director should be in tune with the spirit of the place and with 
its aspirations. 

For several years, I chaired a chancellor's committee on long-range planning. At one meeting, 
when we were reviewing data on instructional costs, one program stood out because its costs 
,>er student or credit hour were about twice as high as those for any other instructional 
program. This was the College of Home Economics. The reasons were obvious: high level 
senior faculty, low student enrollment, and high costs for equipment and laboratory work. We 
also knew that just down the road about forty miles Cornell University had a large, nationally 
distinguished, state-supported college of home economics. The "facts'* su.cN suggested 
abolishing the program at Syracuse. But "data" are seldom sufficient for administrative decision- 
making, because the data do not include personalities and politics and history. From this and many 
other experiences like it over the years I have developed some caution about associating institutional 
research too explicitly with administrative decision making. I prefer to associate institutional 
research wuh learning, 

A few years after I joined the faculty at UCT.A, Chancellor Franklin Murphy asked me to 
review the work of the Office of Institutional Studies and recommend what should be done 
next. The \ice chancellor appointed an advisor) committee to work with me on the 
assignment. Clearly, there was an urgent need to create an integrated information system and 
to operate it in such a way that its power and utility would be sufficiently obvious to ensure 
its continual use for planning, quality control, projections, and other relevant analyses. We 
rvcommendeJ that the office responsible for developing and operating information systems be part 
of a larger sUtffor office concerned with planning organization, and operations. The office that 
was subsequently created has evolved over the years with a high level of capacity, 
sophistication, and service under the direction of Adrian Harris. Another recommendation, 
which I favored but was not regarded as feasible by the committee, was to have a parallel 
office of higher education research an 'I evaluation. I felt that ongoing evaluation studies 
would be a healthy balance to the accounting activities of the planning offi-e. What I 
encountered was a deep-seated belief at UCLA that research was exclusively a faculty 
enterprise. Evaluation was simply a staff or sen ice enterprise. So a parallel or loosely 
connected evaluation and research office wns never treated. 
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Ill all these stories about my personal encounters with in stitutional research and evaluation, 
there has been an underlying concern with what I would like to call the "human side" of 
institutional studies. When we develop our more sophisticated measures, or indexes, or 
indicators, we may be tempted to thinkof them as impersonal and objective. But they are not. 
They are, at bottom, descriptions of human behavior. I looked at the titles of the now more 
than sixty issues of New Directions for Institutional Research and I think all of them had 
something to do with human behatior. 

In all ca\es what's important is how we define the variables. For example, about ten years ago I 
constructed, pretested, and produced a set of measures of the amount, scope, and quality of 
effort students put into using the resources and opportunities college provides for their 
learning and development. This concept, "quality of effort," has now become a major variable 
in new research on student outcomes. I think of a questionnaire as a form of test. What are 
you measuring? And with what degree of reliability and validity? My involvement in 
educational measurement over many years leads me to emphasize my firm belief that the value 
of answers to a questionnaire does not depend on the quantity of respondents; it depends primarily 
on the quality of the questions. 

It is my impression that the last three decades of professional development in AIR have 
emphasized the importance of accounting and computer progra ming for creating &<id 
oper .ting a knowledge network about the institution. Institutional r^earch is one important 
\y * t of a larger domain I would describe as higher education research and evaluation. These 
two interests have, to some extent, led separate lives. I think it is true that the major centers 
foi the study of higher education have little or no connection with the institutional research 
ofTices at their home institutions. Whether various connections are feasible organizationally 
depends on the traditions of the individual institutions. One can, however, encourage the 
steps that have been taken to facilitate conversation and other contacts between AIR, ASIIF 
anil Division J of AKRA. 

In such contacts and connections I would note the importance of philosophers and historians 
because I beliexe that educational policy is best understood in the context of history and 
values. One should know what a college is, and how it evolved over a long history. What a 
college does, of course, reflects the society that supports it. We have a vast number of 
specializations because science required them. We have a vast number of vocational training 
programs because the society needed skilled worker* and professionals. When the Soviets 
orbited a satellite we poured money into the training of engineers, scientists and 
mathematicians. And now, since we don't have any money, we are worried about the Japanese 
and the emerging European market, so we need more people trained in business. We are, they 
say, hung in a global economy, and we must le:^n to compete successfully in it. There is 
abundant political anil patriotic support for this purpose, and if it becomes the dominant 
wilue or bottom line for higher education in the next decade or so, it will influence *v hat is 
olTered, how it is offered, and who pays for it. It will also determine how and with what 
measures we judge the , ogress and the benefits of higher education. But suppose this is not 
the only bottom line. Suppose that the quality of higher education and its contribution to our 
future docs not he primarily m preparing people to compete successfully in a global economy, but 
rather m preparing people to live responsibly in a global community. If the bottom line is not the 
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bank balance in the nation, but the balance of people and resources in the world, then AIR and 
others will need to create a new set of indicators to measure our progress and performance. 

If one of the functions of AIR is to stimulate debate, discussion, analysis, and introspection 
about the operations of our institutions - their purposes and programs, their policies and 
procedures - and if that function is pursued vigorously, then the simple fact that AIR exists 
is good reason to hope and believe thtit higher education will continue to be adaptive and 
effective. 
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Introduction 

Many practitioners of this specialty, see institutional research as 
an applied profession with an emphasis on data handling and 
policy analysis for higher education institutions. According to 
Fincher, institutional research in the 1980s "is best described as a 
professional, technical specialty with strong resources and 
capabilities for policy-related research In institutions of higher 
education" (Fincher, 1985). Peterson and Corcoran conclude that 
institutional research has developed into "a primarily management- 
oriented, applied, data-handling, analysis, and research function" 
(Peterson and Corcoran, 1985). 

I will argue that as an applied profession, institutional research lacks a scientific base. I will 
a\so argue that institutional research is confronted with the fundamental choice either to 
develop such a scientific base for itself, or to drift away from the fundamental values that 
belong to the academic attitude to which institutional research is oriented. 

My personal preference would be to accept the challenge to develop a scientific base. A 
scientific base will increase the legitimacy and the academic standing of institutional research. 
However, we should not forget that science cannot, by definition, offer an absolute and final 
suggestion on hov to proceed in a practical context. As an important epistemological 
argument (discussed below) states, a scientific base can never provide a definite reason to 
design and implement practical solutions. 

There is, of course, another viable option-to limit the activities of the institutional researcher 
to collecting and analyzing data on the performance and the environment of a higher 
education institution. But this option worsens the threat that scientists within higher 
education will not regard institutional research as a profession equal to their own (scientific) 
activities, and this may impair the legitimacy of policy suggestions developed by institutional 
researchers. 

When institutional administrators with limited authority are forced to seek the cooperation 
of the scientific professionals within their institutions to implement institutional policies (as 
is the case in Western Europe), the low academic standing of institutional research may create 
legitimacy problems. Institutional administrators will have difficulty convincing the scientific 
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professionals within their institutions that the policy suggestions developed by institutional 
research are worthwhile. Instead, these policy suggestions may be brushed aside. Scientific 
professionals will argue that the ideas of institutional researchers cannot be accepted because 
they lack the stringency of scientific analysis. 

Can Institutional Research Develop into a Science of Higher Education? 

Already in its early days, institutional research was confronted with the fundamental question 
of whether it could develop into a science of higner education institutions (Dyer, 1966). 
According to Dyer, institutional research "had nowhere to go if it remained purely operational" 
(Fincher, 1985). Institutional research should, instead, try "to fit the problems of particular 
institutions into some sort of evolving generalizations" (Dyer, 1966). 

Fincher notices that Dyer's goal to develop a science of higher education institutions has not 
been achieved so far in the history of institutional research. 'The efforts of institutional 
research to solve institutional problems and to study internal processes are not guided by a 
conspicuous network of hypotheses and conjectures that could be called a theory" (Fincher, 



Fincher also claims that the object of study of institutional research "cannot easily be 
described by nomothetic or lawlike characteristics," and that institutional researchers are 
better able "to analyze and interpret specific events and processes within their institutions 
than ... to explain complex institutional behavior in theoretical terms," His conclusion is 
that although "a more positive attitude toward theory-related research would accomplish much 
in the continued development of institutional research, ... the merits of institutional research 
depend not on its scientific underpinnings but on its relevance and influence in decision and 
policy making" (Fincher, 1985). 

Fincher is optimistic about the future of institutional research, but he may be too optimistic. 
Taking into consideration that most institutional researchers are skeptical about developing 
institutional research into a science, I think the future of this professional field might be 
characterized by a decrease of legitimacy. The suggestions institutional researchers make in 
the context of decision and polic> making may be regarded with increasing indifference and 
e\en mistrust. 

Because it lacks a scientific base, institutional research may in the future be considered a 
management-oriented data collection activity that has little to offer in terms of real scientific 
analysis. Its relevance and influence in institutional decision making could therefore decline 
rapidly, especially as academic disciplines with a theoretical basis (like organization theory 
and economics) start to study higher education institutions more seriously. 

Peterson and Corcoran (1985) suggest that the development of institutional research requires 
an integrative framework in which various disciplinary insights are combined. I am convinced 
that this framework can only be designed when institutional researchers accept the challenge 
to approach their object of study with a scientific attitude. 
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Is Institutional Research a Successful Profession? 



Many institutional researchers consider their field of study a profession, and are apparently 
convinced of the success of their professional activities and outcomes. In this they may, 
however, be somewhat naive, for the literature on the status of professional expertise makes 
it clear that the professions in general are presently confronted with a crisis of confidence and 
legitimacy. 

During the 1960s, the professions were held in high esteem. The professions were expected 
to define and solve all our problems, be they in the field of health, crime, city planning, social 
work, or education. In 1963, the journal of the American Academy of Arts and Sciences, 
Daedalus, devoted a special volume to the professions, proclaiming the triumph of the 
professions in American societ>: "the day is coming when the 'knowledge industry' will occupy 
the same key role in American economy that the railroad industry did a hundred years ago.... 
America has become more cognizant of the professions and more dependent on their services, 
than at any previous time in our history. Thorstein Veblen's sixty-year-old dream of a 
professionally run society has ne>er been closer to realization" (Lynn, 1963). 

Hie 1980s show quite another picture. Confidence has disappeared. Several problems and 
trends (the deteriorating cities, the growing ecological crisis, the increase of poverty, the rising 
costs of medical care, the decreasing quality of education) seem to be related to the 
professional practices that were called upon to resolve them. The definitions and solutions 
advocated by various groups of professional experts have appeared to be ineffective. The 
solutions presented by the professionals have had unanticipated consequences, sometimes 
Horse than the original problems (Schon, 1983), 

As a result, the professional claim to a monopoly of relevant knowledge has been challenged. 
Professionals have failed to live up to the expectations they created. In fields as diverse as 
city planning, housing, criminal justice, welfare, and transportation, policy suggestions 
designed and advocated by experts often du, not solve the problems they were addressing. 

An important reason for this failure is thut the solutu were often derived from poorly 
defined theories or else were not related to theories at « Many professionals now argue that 
their knowledge is mismatched to the practical tasks they are supposed to accomplish. This 
gap between professional knowledge and societal expectations is one that Harvey Brooks had 
already noticed in 196/: The dilemma of the professional . . . lies in the fact that both ends 
of the gap he is expected to bridge with his profession are changing so rapidly: the body of 
knowledge that he must use and the expectations of the society that he must serve" (Brooks, 
1 976). 

Chen the crisis of confidence in the professions, the complacency of institutional researchers 
concerning the professional status of their field may not only be somewhat naive, but also self- 
destrttcthe. If institutional researchers fail to recognize that they are confronted with the 
same challenge regarding the relevance of their knowledge base as other professionals, they 
may soon suffer a complete loss of legitimacy. To avert such a disastrous surprise, it is time 
f..r institutional researchers to analyze the status of their professional knowledge. 
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The Empirical-Analytical Approach of Social Science Research 



In this section, I sketch the empirical analytical approach of social science research. I am 
convinced that this approach should b* the scientific base of institutional research, as it 
should be of most other forms of research in the social sciences. Unfortunately, because 
institutional researchers have paid little attention to this approach, there has been little 
discussion regarding the knowledge base of institutional research. 

The empirical-analytical approach of social science research should not be confused with an 
uncritical imitation by the social sciences of what sometimes are supposed to be the distinctive 
characteristics of the natural sciences. Such an imitation would, as Medawar rightly argues, 
lead to an "unnatural" scientific approach characterized by the wrong belief "that 
measurement and numeration are intrinsically praiseworthy activities; by the equally wrong 
belief that facts are prior to ideas and that a sufficiently voluminous compilation of facts can 
be processed by a calculus of discovery in such a way as to yield general principles; and by the 
uncritical faith in the efficacy of statistical formulae ... the use of which is in itself 
interpreted as a mark of scientific manhood" (Medawar, 1984). 

lhe empirical-analytical approach of social science research has little to do with such an 
unnatural scientific attitude, although the critics of the empirical-analytical approach 
sometimes seem to think so. These critics incorrectly claim that the empirical-analytical 
approach is'positivistic" and "reductionistic," implicitly assuming that the empirical-analytical 
approach indeed is a simple imitation of some of the methods of the natural sciences. 
Moreo\er, they show little understanding of the philosophical and epistemological 
underpinning of the natural sciences. As I will indicate below, this critique misjudges the 
leading principles of the empirical-analytical approach of social science research. 

Tlieory design. What then are these leading principles? The first principle concerns the 
obligation to design theories properly. Theories are the \e hides by which we try to grasp the 
complexity of the world around us. When designing theories we reduce the complexity of 
reality. We assume certain regularities and we also omit other regularities, which allows us 
to formulate our assumptions. As Popper has stated, designing theories, in this sense, can 
be characterized as "the art of systematic over-simplification--the art of discerning what we 
may with advantage omit" (Popper, 1982). 

A crucial quality of the theories we use to try to understand realty is that these theories 
should be falsiflable. A tlieory that is not falsiflable does not provide information, simply 
because, although it may always be in accordance with the facts, it does not indicate when it 
ma> be supposed to hold and when not. A statement "It will rain or not rain here tomorrow" 
may be correct, but it cannot be refuted and it does not give us information. Theoretical 
statements that cannot be incorrect when confronted with reality do not help us to understand 
reality. 

Another quality of our theories concerns their internal consistency. Theories should be so 
designed that one cannot deduce from them both a statement about reality and the negation 
of that statement. To refer again to Popper: "A consistent sjstem . . . divides the set of all 
possible statements into two: those which it contradicts and those with which it is compatible 
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(Among the latter are the conclusions which can be derived from it). This is why consistency 
is the most general requirement Tor a system, whether empirical or nonempirical, if it is to 
be of any use at all" (Popper, 1974). 

The requirement to design theories that are internally consistent holds (as can be seen in the 
citation from Popper) for all theoretical systems, whether they are normative, mathematical, 
or empirical. For empirical theories, moreover, another requirement Is of extreme 
importance: the requirement to design theories that also are externally consistent This 
requirement implies that statements about observable phenomena that can be deduced from 
theories should not contradict the observations. When such contradictions are found (and 
cannot he removed), the theoretical system from which the statements are deduced is refuted, 
and must be replaced by another, better theory. 

Accepting the fabijicationist approach. The above mentioned considerations indicate the second 
principle of the empirical-analytical approach of social science research. This second principle 
includes the acceptance of a falsiflcationist approach to social science research: when a theory 
is found to be external!) inconsistent, it is judged to be falsified, and we should try to replace 
it 1>\ another theor). 

The second principle also implies the rejection of instrumentalism, the doctrine that scientific 
theories are no more than tools to enable us to act successfully. The instrumentalist position 
allows the acceptance of a theory that is internally or externally inconsistent, as long as it is 
applicable. Such a position is not acceptable for those who take the empirical-analytical 
position, for their ultimate criterion is not the applicability of a theory but the endless search 
for truth (Popper, 1983). 

The two first principles mentioned abo\e allow us to formulate two further principles on how 
to proceed when doing social science research. As I indicated before, a theory that accords 
with the facts hut does not specify uhen it is and is not supposed to hold is not informative. 
Such a theor) is said to ha\e an empirical content of "zero": it is not possible to deduce from 
it one single statement that could be confronted with statements about observed phenomena. 
Along the same lines, it should be self evident that the more a theory excludes (i.e., the more 
statements about observed phenomena that are not "allowed" by the theory), the better the 
theory is (i.e., the more the theory sa)s about the world around us). Such a theory has a 
large empirical content. 

Maximize empirical content. The third principle is that social science researchers should try 
to design internally and externally consistent theoretical systems that have as large an 
empirical content as possible. Social science researr!:ers should strive to formulate theories 
of which the class of possible falsifiers is as large as possible. 

L niversality. The fourth principle can be directly related to the third. In their striving to 
formulate theories with a large empirical content, social scientists should try to deduce from 
these theories strictly universal hypotheses that can be confronted with reality. These 
universal hypotheses should be derived from a consistent theory that should also specify under 
what conditions the h>po theses are supposed to hold. Universal hypotheses are far more 
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interesting than singular or "particular" hypotheses, exactly because the class of possible 
falsifiers is much larger when uni\ersal statements are used. 



Together the four principles of the empirical-analytical approach of social science research 
indicate that empirical theories are the best mechanisms we have to try to understand reality. 
Because empirical theories are formulated as internally and externally consistent systems of 
universal statements, and because they are designed in such a way that their empirical content 
is as large as possible, we may expect that these empirical theories offer us the best possible 
intellectual gH« of the world around us. Thus, if we want to try to solve problems in our 
environment, we should try to make use of these theories. 

The Professional Knowledge Base of Institutional Research 

According to several authors, professional knowledge is defined by a number of characteristics. 
Moore, for instance, has argued that the two primary bases for specialization within a 
profession are first, the substantive field of knowledge that the specialist professes to 
command; and second, the technique of production or application of knowledge over which the 
specialist claims mastery (Moore, 1970). 

Similarly, Schein mentions three crucial components of professional knowledge: first, an 
underlying discipline or basic science component upon which the practice rests or from which 
it is developed; second, an applied science or engineering component from which many of the 
day-to-day diagnostic procedures and problem-solutions are deri\ed; and third, a skills and 
attitudinal component that concerns the actual performance of the client, using the underlying 
basic and applied knowledge (Schein, 1973). 

These authors indicate that the knowledge base of a profession is to be found in basic science, 
or at least in a scientific body of knowledge about the field with which the profession is 
concerned. Without such a scientific base, professions cannot laim to be more than arbitrary 
approaches to specific aspects of reality. Without a sciemific base, professions hick the 
legitimacy to impose their solutions upon the practical problems they want to address. 

As mentioned previously, scientific knowledge has to be understood as the collection of 
empirical theories that are formulated as internally and externally consistent systems of 
uni\ersal statements with as large an empirical content as possihie. A profession based on 
such a collection of theories has at its disposal a general hody of knowledge, which is 
formulated in universal terms. Professionals can apply this general knowledge to specific and 
Concrete circumstances. "Professionals apply . . . general principles standardized knowledge, 
to concrete problems" (Moore, 1970). 

Glazer has made a distinction between "major" and 'minor" professions. Major professions 
are grounded in systematic, fundamental knowledge: minor professions lack this foundation. 
Minor professions are threatened because the academic disciplines from which a foundation 
could perhaps be developed are superior in status to the professions themselves. The 
prototypes of the major professions are medicine, law, and engineering. Examples of minor 
professions include sotial work, education, and town planning (Glazer, 1974). 
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I fear that institutional research has to be classified as a minor profession. Institutional 
researchers so far have not been able to find a collection of empirical theories that they could 
use as a general body of knowledge for application in specific circumstances. Even worse, 
institutional researchers apparently think that their Profession should not be founded on basic 
science, but on the techniques of data collection and data analysis. Institutional researchers 
often claim that their profession has to be seen as a data-handling function based on 
specialized data-handling techniques. 

This inclination to be satisfied with the status of a minor profession, or to even deny the 
importance of a real scientific base, seems to be the fundamental threat that institutional 
research will have to face in the future. If institutional researchers keep on convincing 
themselves that the elegance of data-handling techniques offers a solid enough basis for their 
profession, they should not be surprised when this profession starts to pine away. If 
institutional researchers do not accept the challenge that has been put forward to a!! 
professions (i.e., the challenge to reconsider the strength of their scientific bast), they perhaps 
will stop being a profession. If institutional research wants to maintain its professional status, 
it must focus on the design and the testing of empirical theories-in the hope that by doing 
so a scientific base can be found. 

The four principles of the empirical-analytical approach of social science research can be 
helpful here. If we want to build up our scientific knowledge, we should use the 
epistemological principles that have proven to be s essful. 

If institutional researchers would embark upon this task of designing and testing theories 
there is one point, however, that should be kept in mind. Empirical theories cannot offer 
anybody in any profession a final argument on how to act in a specific practical context. 
Empirical theories provide us with mechanisms to understand reality. However, such an 
understanding is reached by assuming certain regularities about reality and leaving out many 
other aspects. Designing theories indeed is the art of "systematic over-simplification". 

The notion that empirical theories (perhaps not yet but one day), will be able to provide exact 
knowledge on how to act in a practical context is referred to as "scientific determinism." The 
general idea of scientific determinism is that the world and humanity can eventually be 
completely understood, and that it is the task of science to acquire the level of knowledge that 
will enable us to predict with complete exactness any future state. 

Popoer has introduced the so-called 'demon of Uplace" to explain why the notion of scientific 
determinism is not correct. "I place believed that the world consists of corpuscles acting 
»pon one another according to Newtonian dynamics, and lhat a complete and precise 
knowledge of the initial state of the world system at one instant of time should suffice for the 
deduction of its state at any other instant" (Popper, 1982). Laplace knew that this kind of 
knowledge is superhuman. This is why he introduced the fiction of a demon: "a superhuman 
intelligence, capable of ascertaining the complete set of initial conditions and the laws of 
nature ... the demon would be able, according to Laplace, to deduce all future states cf the 
world system" (Popper, 1982). 
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Laplace, we now know, was wrong in assuming a deterministic world. Epistemology now 
teaches us that we will never be able to establish all relevant initial conditions and to 
formulate ail relevant theoretical laws, to formulate exact predictions. As Popper has said, 
we could never predict the creation of a work as Mozart's G minor symphony, however well 
we study Mozart's brain, paper, pen, and environment (Popper, 1982). Our theories are 
simplifications of an endless complex reality, and they can only be mutually compared. Our 
theories, moreover, are attempts to try to find the truth by testing what is not true. But truth 
remains an ideal. If we would reach it, He could not know it. 

Professionals w ho might think that a scientific base for their profession could offer tiiem exact 
prescriptions on how to act in a practical context would be making the same mistake as 
Uiplace. This belief disregards the epistemological principle that theories, by definition, are 
simplifications and hence can never offer exact predictions or prescriptions. When one 
deduces from a theory certain prescriptions on how to act in a practical context, one will 
always be surprised by the initial conditions not addressed in the theory; one will always be 
confronted with unexpected consequences. 

The lesson to be drawn from this is th.it professionals should be modest when basing their 
applications on empirical theories. The) should be willing to learn from changing 
circumstances; and they should try to design solutions that are flexible, leaving open other 
options if a first one proves to be ineffective. 

Such a modest professional attitude conies \er> close to the characteristic approach of what 
Popper has called "piecemeal social engineering/' A piecemeal social engineer knows how little 
he knows. "He knows that we can learn only from our mistakes. Accordingly, he will make 
his way, step b> step, carefully comparing the results expected with the result achieved, and 
always on the look-out for the unavoidable, unwanted consequences of any reform" (Popper. 



The future of the profession of institutional research, I hate argued in this paper, nun hold 
the unpleasant surprise of a loss of legitimacy Such a loss nun result from the present-din 
preoccupation with the techniques of data handling. While all professions are confronted with 
a crisis of confidence, and while they are all being challenged to reconsider their knowledge 
base, institutional researchers apparently are satisfied with the notion that their activities are 
undertaken with a certain technical elegance. 

If institutional researchers decide to counter the threat that their profession may pine away, 
they will have to accept the task of finding a scientific knowledge base. They will have to 
make use of the empirical analytical approach of social science research, accepting the 
principles that are included in this approach, but realizing that e\en a scientific knowledge 
base cannot provide final solutions for practical problems. 



1969). 



Conclusion 
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Fran the PM for the Future 

(presidential session: part one) 




Gerald W. McLaughlin 

President 

Association for Institutional Research 

(This is an edited record of an address presented in the Grand Ballroom of the Gait House 
in Louisville, Kentucky, on Wednesday morning, May 16, 1990.) 

AIR traces its beginnings to ideas expressed at an Institute on 
Institutional Research luncheon held by the Southern Regional 
Educational Board on July 14, I960. A group of professionals 
interested in institutional research followed up by holding a 
meeting the following year in March at the Morrison Hotel in 
Chicago before the conference of the Association for Higher 
Education. 

The earliest years of AIR were marked by a specific intent to 
remain small and informal. This, then, is the first point that I 
want to make: our association is built upon the atrength of 
friendships, understandings, and interpersonal relationships, We 
started as, and still are, people who like people. The networks and bonds we have forged over 
the years are fully as important as our specific body of knowledge. The paradox Is that the 
forum now regularly attracts over one thousand members, and close friendships among all 
members are impossible. In addition, as we seek to provide services to those who perform the 
institutional research function, we continue to grow in membership~as we should. 

Within this organizational framework, the regionals with their smaller meetings become the 
source for many lasting friendships. Since the costs of attending regionals may be leas, they 
offer an opportunity for many entering professionals to learn about institutional research 
(IR). Regionals are also a place where we can review how IR fits in with related disciplines, 
since each regional develops a discipline focus based on its specific situation and the 
combination of issues within its region. 

We need to become more involved with the linkages we have with our regionals and in the 
exchange of mutual support. While all of us have had the opportunity to attend this forum, 
we should also be involved in our regional activities and in collegia! networks that link us to 
colleagues beyond the direct institutional research function. 

It seems to me that the second point inherent in our first 30 years as we come of age is that 
we never have had a clear understanding of what we do for a living. Arthur Adams, president 
of the American Council on Education in 1960, has noted that: Those actively engaged in 
institutional research are, naturally, proponents of its value. They are not entirely agreed, 
however, on its scope or functions" (Brumbaugh, 1960). John Stecklein (1966), when president 
of AIR in 1966, noted that we were divided between the need to do "intensive, theoretically, 
oriented, long-term research" and the need to be "an extension of the president's or executive 
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vice president's office with the primary goal of finding how to use financial resources to better 
advantage". 1 think we should be proud of the fact that we are both institutional and research. 
This debate continues today and I am convinced that it should. 

It is a strength of our association that we work to discover results from research and to apply 
these truths to the nuts-and-bolts problems that limit the quality of our institutions. We need 
to identify the information needs of our institutions and to undertake research that provides 
better information in support of management, policy analysis, and planning. 

With our constantly changing and expanding knowledge, we need to provide for both the 
theoretical and the practical aspects of our profession in our meetings and in our professional 
development. As for our publications, we need to sustain the excellence of our research 
journal, Research in Higher Education, while we publish more practical applications of IR. 
This expansion may involve strategies such as expanding the Professional File or creating a 
new publication. 

The third issue I would like to discuss, as I consider our first thirty years is that 
professionals in our field have always wanted to be leaders in improving the quality of our 
institutions and higher education in general. We have often looked to our college and 
university presidents and others for indications that we were making a difference, that we were 
studying the correct issues. We have essentially become part of the middle management of 
our institutions, but that does not uean that we are not also important leaders in our 
institutions-quite the contrary. We have greater opportunities to be leaders than most 
because we work with information about issues critical to postsecondary education This 
information, if it is reliable, relevant, and timely, makes for an extremely strong power base. 

We cannot wait, however, for others to come and ask us to be leaders; we need to know our 
institutions and seize opportunities as they arise. We must develop and use leadership skills 
in our institutions. 

It is easy to talk about leadership, but we should be aware of serious future challenges in our 
profession. Two issues identified by the transition committee are directly related to these 
challenges: 

1. Financial pressures on higher education institutions may well restrict the central support 
functions of middle management to include many of the functions performed by our 
members. 



2. 



Computer technology will increasingly allow administrators to create and directly access 
their own data, reducing their reliance on others for analjses. 

These challenges require that we not only become leaders and competent managers, but that 
we also develop a base of interpretive knowledge relevant to our institutions. I am convinced 
that of the multitude of general issues in higher education, the most critical underlying need 
.s to improve the quality of our institutions. This is consistent with our function to sustain 
quality operations between self-studies. It is consistent with our purpose to "benefit, assist, 
and advance research leading to improved understanding, planning, and operation of 
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institutions of post -secondary education" (Article II, AIR Constitution). My conclusion is that 
He need to learn more about becoming better leaders and about the available strategies for 
improving the quality of our institutions. 

Let me summarize my three points and relate them to why I invited Larry Sherr to return to 
the Forum to talk to us today. 

1. I h association has a strong basic people orientation»"high tech with high touch" if you 
agu nith Nesbitt. I-arry proceeds on the premise that people play an important ro l e In 
achieving quality. 

2. This association Las a sense of purpose accompanied by a constantly changing and 
expanding body of knowledge. Both the practitioner and the scholar have made 
contributions to institutional research. Larry will talk about improving quality as a 
never-ending process and provide information that transcends and applies to the breadth 
of our specific functional areas-information that has been established and documented 
as practical in numerous studies. 

3. Third, the members of this association are sensitive to the need to be leaders end to deal 
with critical issues in their institutions. I believe that quality in our institutions is a 
critical issue and that we need to become leaders in developing quality. Larry will talk 
about how to improve quality without spending more money; that is a message that is 
always of interest. 

Larry, tibove o ia teacher with a vision on how to achieve higher quality in our colleges and 
universities, i am delighted to have him here to share his vision with us as we prepare for 
the next thirty years. 



Brumbaugh, A. J. (Kd.) (i960). Research designed to improve institutions of higher learning. 
Washington, D.C.: American Council on Education. 

Stecklein, J.E. (1966). The birth of a profession. In C.H. Bagley (Ed.), Research on Academic 
Input. Proceedings of the Sixth Annual Forum of the Association for Institutional 
Research, pp. 9-13. 
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Quality i» Higher Education- A Verb, Not a Noon 
(presidential session: part two) 

Lawrence A* Sheer 

Chancellors Club Teaching Professor 

and Professor of Business 
University of Kansas 



(This is an edited record of an address presented in the Grand Ballroom at the Gait House 
in Louisville, Kentucky, on Wednesday morning, May 16, 1990.) 



Institutional researchers need to play a new role to help 
universities and colleges improve the quality of their programs. 
Your current role of gathering, collecting, analyzing, and 
distributing information is an important and continuing one. Now 
you need to become problem consultants. You ar located in a key 
spot in your organization, and have the km Jge to identity 
problems and the ability to suggest an array of alternatives for 
improvement. 

Many institutions are using a management system based on a 
failed philosophy. We have imitated America's largest 
corporations in the way we administer ourselves-adopting such 
things as levels of management (i.e., supervision and a formal chain of command). We have 
forms, reports, and data, much of which we do not need or use. We have checks and 
approvals (i.e., inspections). We have elaborate and formal evaluations that \tty frequently 
have no constructive basis. We have competitive bidding that gets u» th* cheapest materials 
but frequently increases our costs. We have elaborate planning systems that often do not 
work well. We have inflexible organizations t t seem to be more in love with procedures 
than anything else. 




These are the r tsons that many U.S. businesses are unable to compete in the global economy. 
There are alternatives. Last year, I presented the work of W. Edwards Deming. I could have 
mentioned others, particularly Joseph M. Juran, whose models have worked better than other 
management models used in the United States in the 1980s. I believe these models will 
continue to outperform other management models used in American industry in the 1990s and 
in the next century. These models have values that are more compatible with higher education 
than the values of the current management system. 



I work in a team with two other University of Kansas professors, Dennis Karney and Steve 
Hillmer. Many of their ideas are contained in this pp**r. Also I want to thank Ellen Chaffee, 
who is responsible for the expression H a verb, not a noun." 
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When we talk about quality, the first thing that conies to mind in both higher education and 
business is money. We can provide or improve quality if only we had more money. Consider 
the people who provide the money. Our revenues come from students and parents, state and 
local governments, the federal government, alumni, friends businesses, and philanthropic 
institutions. In business, these sources are called customers or perhaps investors. Educators 
do not seem to like these words. Alternatives include clients, consumers, patrons, benefactors, 
buyers, purchasers, shoppers, and even end users. Perhaps we should call them "those folks." 

It is "those folks," plus some other users of our products and services including employers, 
other academic institutions, and academic peers, who must be the focus of our discussion, 
hi the past year, almost e\ery leader in higher education has made a statement claiming that 
"those folks" are not providing us with enough money. 

What does the word "quality" mean to you? Consumers describe quality by the characteristics 
of the product or service: it works; it is durable; it is available. We want good service, 
courtes>, and value. American consumers have spoken very loudly and said that the cheapest 
product does not necessarily pro\ide lasting \alue. One way to define quality is by the output. 

Quality has three dimensions: design, output, and process. Design refers to what the output 
is intended to he. A slide rule would be a current example of a product that might be of high 
quality in the output dimension hut not sell today (i.e., low quality design). These properties 
must be considered from a customer perspective. Design is what we intend to do. Output is 
what comes out of the system. Process is how we do it. Very simply put, inputs are coming 
in. outputs are coming out: what goes on in the miudle is the process. 

Process is not given appropriate attention. Now, some say we pay too much attention to 
process or procedures. I myself have often complained that following the correct steps seems 
more important than the end result. On many campuses, affirmative action programs would 
be an example. So when I say we do not pay enough attention to process, I am saying we 
should question whether or not a process is appropriate for the task to be performed. 

Consider some of the characteristics of a process. Sometimes it is necessary to fix an earlier 
mistake. I lus happens, for example, with enrollment, payroll, even teaching. We have a word 
for fixing earlier mistakes in business: "rework." Another operating characteristic of these 
processes is giving up on something and starting all over again. The same examples are 
appropriate. This is called "scrap." In other procedures, some of the steps are unnecessary. 
That is "unnecessary complexity." Although many of us feel that we have unnecessarily 
complex procedures imposed on us from outside, we are clever enough to add our own layers 
of unnecessary complexity. 

Consider the following example. The payroll process in a large state university was not 
working well; errors and complaints were common. People blamed everyone but 
themselves. The payroll office blamed the people who initiated the forms. The university was 
decentral.zed and over 300 locations (people) initiated transactions. No one even had a list 
of the names of these people. The people who initiated the forms blamed the payroll office. 
Data were collected. 
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It was discovered thn' 50 percent of the forms were either not completely filled out or were 
filled out incorrectly. This is not very surprising; the university had no required training 
program for these people. Ninety percent of that 50 percent could be corrected by making a 
phone call, but this was not as easy as it sounds. Rework. Ten percent of the forms were in 
such bad shape that they had to be sent back to be started all over again. Scrap. A flowchart 
of the process was drawn, and it was discovered that depending on where 0 transaction began, 
there were between three to seven inspections (i.e., approvals and signatures). 

The mistakes were studied using a Pareto analysis. It was discovered that 50 percent of the 
appointments involved student help &nd that these transactions accounted for 70 percent of 
the errors. Therefore, we focused our attention on the student appointment process. Many 
believed that the inspections were adding cost and not adding value, and it was proposed that 
they be eliminated. 

After considerable debate, the experiment proceeded, but unfortunately the people who 
initiated the transactions were not told that the inspection process was eliminated. There 
were "naysayers" who said that by eliminating the inspections the error rate would increase. 
The rate appeared to decrease slightly and the time it took a transaction to reach the payroll 
office decreased by forty-eight hours. Costs went down. Imagine what a good training 
program could accomplish. 

This story has another chapter. State employment policy says that every classified employee 
has to attend an orientation program within 21 days of employment. That rarely occuired. 
Analysis of the flowcharts revealed that the personnel department was not informed about new 
hires until after the payroll office approved them. Data revealed that, on average, the 
personnel department received that information 17 days after the employee was hired (the 
standard deviation was 11!) What was the first proposed solution? Add another copy to the 
form. 

Here is a process with rework, scrap, and expensive steps that added no value, just cost. What 
was the proposed solution? Make it more complex! 

It is important to understand that these process characteristics add cost and simultaneously 
loner quality. These things go together. As the Finding and Purposes section, of the Malcolm 
Baldridge National Quality Award, established in 1987 in the U.S. states, "American business 
and industry are beginning io understand that poor quality costs companies as much as 20 
percent of sales revenue nationally, and that impr^.eu quality of goods and services goes hand 
in hand with imprrved productivity, lo' <t costs, and increased profitability" (Public law nos. 
100-107). 

As a percent of cost of goods sold, which is a more appropriate comparison than sales 
revenues for nonprofit higher education, the cost of poor quality is greater than 20 percent. 
K\en if we are administrating our institutions considerably better than industry does, which 
I seriously doubt, it would he worthwhile to study our processes as well as our designs and 
outputs. 
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Consider what we do in higher education. Accreditation has historically focused on inputs. 
Assessment is moving into outputs, and that is a good direction. Faculty members usually 
concern themselves with curriculum changes (i.e., design). Except for people like Peter Ewell 
who emphasize the assessment process, as opposed to simply assessment, few people are 
studying processes. 

Continuous process improvement is m> subject today; quality is a verb, not a noun. It has 
five key ingredients; honesty, shared vision, patience, commitment, and a theory or plan. 

Honesty 

You cannot solve a problem unless you admit that it exists. You cannot improve a process 
unless you admit that there is room for improvement. We have many problems in higher 
education; the media has pointed this out often enough. Common issues include the poor 
preparation of students in grades K through 12, racism, sexism, cheating, the availability of 
courses, advising, the quality of student life, and the expansion of administration and support 
sen ices personnel at a rate considerably higher than enrollments. 

We must recognise that some of the problems are of our own making. We do not have to 
apologize for this, although that would not be a bad idea; we just have to admit that we are 
responsible. That is a very important observation. If we are responsible for a problem, then 
that means we have the power to do something about it. And that is a fundamental 
observation. I am not interested in who is at fault here. I am interested in who has the 
power to improve the situation. 

Some problems are not of our own making, but by ignoring them, we not only seem to 
condone the issue, we make the situation worse. Why are we reactive? Why not be proactive? 

Imagine a business denying that their products or services are of poor quality but saying, yes, 
the) could improve quality if they only had more money; they will raise the prices of their 
products and use the money to provide better qutility in the future. This is the way many 
interpret our appeals for more money. 

NNe must admit that problems exist and that we have the power and responsibility to improve 
the situation. 

Shared Vision 

Continuous improvement requires that etenone be involved. We need a shared vision in our 
communities that includes quality --not just design and inputs, but also outputs and process. 
We need leadership to accomplish this. 

Patience 

If a significant portion of our resources are currently being used unwisely, they cannot be 
fcuinJ ou the day you must retrench. Budget cutting often leads to a lowering of the quality 
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of outputs. Raising quality by process improvement takes time and study. There is no magic 
wand, and patience is required. 

Commitment 

The continuing improvement process requires that everyone in our community is committed 
to improvement. The chancellor, tht president, the administration, the faculty, the students, 
and the staff all have important roles to play. However, we do not have to wait for everyone 
to buy in before we can begin, we can start in our offices today. 

A Theory or Plan 

The theory or plan also has five points: customer (or mission) focus, systematic approach to 
operations, vigorous development of human resources, long-term thinking, and once again, 
commitment. These points form a complete system. You cannot fully understand any one 
point until you understand all five. 

Customer (or mission) focus. By studying success stories in industry, we find that virtually 
every successful company has a driving desire to focus on satisfying customer needs. These 
companies are willing to change processes and designs to achieve better outputs at lower costs. 
Successful organizations are flexible. 

Ours are often inflexible. Derek Bok, speaking to Harvard's Board of Overseers in response 
to public criticism of higher education, said: 

Most of the charges are flawed because they ignore basic conflicts and contradictions in 
the demands society makes on universities. ... By ignoring the conflicts that underlie so 
many of the complaints, the debate will remain superficial. ... In particular, we need to 
step back and ask whether our universities are doing all they might to help the country 
address its most important problems-lagging competitiveness, poverty, inadequate public 
education, environmental hazards, and many more. (Chronicle of Higher Education, April 
18, 1990, A2) 

This is a call for customer focus, a call for a need to know our mission. We need to take 
another hard look at our mission and then do something about it 

Peter Li kins, president at l^ehigh, writes that we need to change our culture: 

Each college and university . . . should ask itself if it needs all its vice presidents and if 
these vice presidents need all their managers. Over the past two decades, there has been 
a tremendous growth in personnel on the U.S. campuses, but it's been primarily outside 
the faculty rank. . . . After we reduce administrators, we need to talk to our faculties 
about consolidating academic units to accomplish our missions more effectively. 
{Chronicle of Higher Education, May 0, 1990, Bl - B2) 
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Tlie administration must be a role model for the faculty. Faculty are going to be very hard to 
change, and if you can come to them with some success stories, they might be more receptive. 
In the same article, Likins continues: 



"We cannot succeed in such efforts if our only goal is to reduce costs, however. Academic 
leaders have to convince their faculties and staffs that managing fewer resources can 
produce something of permanent value, rather than simply causing bloody budget cuts." 

Businesses have learned this. One business after another has found out that cost-cutting 
efforts often lead to cutting not only the unnecessary but the necessary. Likins believes that: 
"Higher education needs to emulate the process that corporate America has been going 
through for the last decade. . . . We have to ask ourself, 'What the hell are we here for?'" 

Mission! We have to rethink our goals and priorities. We have to know what our mission is 
so we can measure our expenditures against our stated purposes. We have to ask ourselves: 
Who are "those folks?" What are we doing for them? What are their needs? We have to 
define these needs clearly to be able to measure our successes and failures. I am not naive 
enough to believe that we must measure everything, but we can make significant steps in that 
direction. We must change our institutions as the needs of "those folks" change. However, 
we have e\ery right to do our best to influence what "those folks" think their needs are. 

Systematic approach to operations. Systematic, not random. We need continuous improvement 
of our procevses as well as improvement of the other dimensions of quality. This requires 
focusing on the customer, the process, and the data. 

B> focusing on the customer, I am now referring not only to "those folks," but also to the 
internal customer. Kach of us is both an internal customer and an internal supplier to other 
customers in the organization. If one of us does a bad job, the next person must either 
rework the task, send it back to ha\e it done all over again, or worse, pass it on to the next 
internal customer. Kach of these choices increases costs and lowers quality. Do you know 
who your internal customers are? Are you satisfying their needs? Can you do better? These 
questions need answers. 

Next, we must focus on the process. What does your supervisor know about your office and 
the process for which you are responsible? He or she probably has a pretty good idea of what 
the inputs are. The supervisor knows something about the outputs, but not as much as you 
would like them to know. Right? But what is known about the process, the way your unit 
transforms the inputs into the outputs? All too frequently, nothing. 

What is going on inside the process? The process can be broken down into six components: 
people, materials, work methods, facilities, machines, and a measurement system, all of which 
take place within a culture. These six items need careful analysis. 

If an employee is not properly trained to do a job, why are we surprised when the job is not 
done well? We make people department chairs because they are excellent professors, but this 
is no guarantee that they can balance budgets or lead other professionals. What kind of 
preparation in teaching do we give to new faculty and graduate teaching assistants? / know 
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of no institution in our society that does a poorer job of educating its own employees than higher 
education. We desperately need continuing development programs (i.e., education not 
training). 

Suppose you have a well-prepared person in a position and the raw materials are poor. Can 
that person do a good job? No. We have competitive bids that all too often guarantee a low 
price for materials but because of scrap and rework drive up our total costs and lower quality. 
Put that same person in inadequate facilities with poorly designed work methods, hold him 
or her responsible for the quantity, not quality, of the job, and you know what happens. Many 
institutions compete with primary and secondary schools for resources rather than working 
with them and thereby help themselves and society. 

We need to be honest. We need to take a fresh look at ourselves and simply ask the question: 
Are we doing our job appropriately for the 1990s and the twenty-first century? We need to 
change our culture to one that encourages everyone in our communities to work toward 
improving processes. We need a culture lhat stops placing the blame on the people, a culture 
that helps people do better. 

We have to focus on data. Properly interpreted, data can tell us a lot. Consider the Shewhart 
Cyde, Plan -Do-Check-Act (PDCA), which is a way to apply the scientific method to process 
improvement. 

Plan. Find an area for improvement. Come up with a theory that says if we change such and 
such, we will see some type of improvement. Do. Run an experiment; try it on a limited scale. 
Check. Collect some data to see if the experiment was a success. Act. If the experiment was 
successful, implement the idea; if not, learn from your mistakes and try again. How do we 
advance knowledge and science? We advance knowledge and science by rejecting our 
hypotheses. Failures are just as important as victories if we learn from them. PDCA should 
be thought of as a continuous cvcle going from one theory and experiment to the next. 

I tench a statistics class to some 300 students. Many years ago, I heard George Box of the 
University of Wisconsin allude to the fact that every process generates the data necessary to 
improve it. Obviously teaching is a process, so I asked the question: What data do I have? 
It always takes a long time to come up with obvious answers. The data that I had (it took me 
about seven months to figure this out) was called my grade book. I found that the 
relationship between homework performance and final exam performance had a coefficient 
determination of approximately 0.6. 

Plan-Ifo-Check-Act. My class meets two hours in a large lecture, and we take a ten-minute 
break midway in the class. Theory. I ask my TAs to give me a homework count during this 
hreak. If the quantity of homework is significantly down then I can urge students to do their 
work for the next class during the second hour. It worked. I had taught the course for ten 
years, and I used to get the homework counts at the end of the semester. Although problems 
remain, by using PDCA I have witnessed substantial improvements. 
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Vigorous development of human resources. There is a preat need for leadership and education. 
How strange that we in higher education do such a terrible job of educating our own people 
in how to run an institution. 

As William A. Goiomski, a quality improvement consultant and lecturer at the University of 
Chicago said, "Managers deal with transactions. Leaders deal with the transformation of an 
organization." That is what education needs-a new culture, one that really believes in 
continuous improvement and practices what it preaches. We all know about managers who 
deal only with transactions; what are they doing to improve the situation? 

Peter Likins does not argue tha* we need more management or even better management. He 
says we need less management: 

There are other positive consequences of managing less. People at lower levels in the 
organization have more responsibility and more freedom to exercise it; the combination 
translates into more power. They need fewer approvals, so they have more autonomy in 
making decisions. They have to trust and respect each other more to get the job done, 
because there is less supervision and more reliance on cooperation. People in such an 
environment must meet high standards of quality, and this means that they also have to 
be well paid. 

Let me add that they also need education to do their job. He continues: 

Communication throughout the organization is more critical because a high premium is 
attached to the effectiveness of cooperation. If we accomplish such changes, we'll need 
to talk to each other more and control one another less. It might be more fun [Ibid.] 

If you look at industry, you will see that this approach works! It adds quality without costs. 
We need an improved environment. We have more fear of the consequences of doing a poor 
job on our campuses than we want to admit. We often hear people say that we should allocate 
funds to departments or faculty who are doing well and not to the others. That is called 
punishment. Many of our campuses fire coaches when they do not have a winning record. 
And as one of them once said: "How would you, professor, like your life to depend on the 
performance of an 18- to 22-year-old?" That's management by fear. Publish or perish! My 
emphasis is on the last word. That's management by fear. Up or out. Fear! Registration. 
Fear! We need to remove that fear so people can perform at their best, We need to develop 
human relations. 



Long-term thinking. Peter Drucker describes it best. Long-term thinking is molding the future 
by understanding the consequences of what we do today. Every day we delay starting the 
change is a day wasted. You can start in your own offices without anyone's approval. Do not 
wait to be asked; begin the journey; be a role model. 

Commitment. I am being redundant on purpose. Quality is not a spectator sport. It is not 
something we can delegate io someone else; it is something with which we must all get 
imoUed. I am not talking about making an incremental change in our organization of only 
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4 or 5 or 6 percent I can give you case history after case history of improvements of 50 to 
100 percent, but let me just mention one case. 



L.ist year I spoke to you about the problems that Motorola had in making televisions back 
in the 1970s. They got out of the business. In 1988, Motorola was one of the first companies 
to receive the Malcolm Baldridge National Quality Award. They learned their lesson, and they 
learned it well. They committed themselves to improving quality by using many of the ideas 
discussed here. Today they are the prime supplier of electronic pagers and cellular telephones 
for Nippon Telephone and Telegraph. Their defect rate today, not only in manufacturing but 
in support services and technical publications, is below 3 in 10,000. They have adopted a goal 
that they call the six sigma goal. They want their processes to perform in such a way that 
their specifications are six standard deviations away from the process mean. 

Implementing the Plan 

Why don't we do it? Every place I go, management's initial reaction is to say this is all very 
nice, but my business or institution is different. Happily, with patience, people change their 
minds. Higher education is also different, and some of the differences make it difficult to 
initiate changes. After ail, I am calling for a revolution, only it has to happen in an 
evolutionary way. Our conservativeness does make it difficult. But consider the values that 
I hiive been stressing. 

The importance of people. When many people hear about cutting costs, unfortunately the image 
that comes to mind is the work of Frederick W. Taylor. I am not proposing an authoritarian 
system. I am not suggesting a system without checks and balances. I am presenting a system 
that empowers people! Less management. You would think that valuing people is compatible 
with higher education. 

The need to use knowledge. I am amazed at how faculty become administrators and do not use 
what they know. Statisticians often do not use statistics; scientists often do not use the 
scientific method. Is it that these people do not respect their own disciplines or that they do 
not respect our processes? Ut us use our knowledge. Universities and colleges are a 
knowledge resource. A good friend of mine, Phil Humphrey, a biologist and director of the 
Museum of Natural History ttfc t!;e University of Kansas, sees the museum as an ecosystem. 
He leads by "constructing hypotheses about people, not making judgments." 

Continuous improvement. Why can't we admit we are not perfect? Show me somebody who 
belie\es that their education is complete and I will show you someone who has turned off their 
mind. Show me an organization \hz* Ho Ca not believe it has room for improvement, and I will 
show you an organization that is in desperate trouble. 

Why don't we in higher education, we who hold dearly such values as the importance of 
people, knowledge, and continuing improvement, why don't we practice what we preach? I am 
convinced that if we do, over a long period of time, not next week, not next year, but over a 
five- to ten-year period, we in higher education can show significant improvements. 
Institutional researchers should and must play a leading role in this transformation. 
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INFORMATION ABOUT AIR in J BI J C ATIONS 



AIR publishes or sponsors a number of items on a regular basis and some types 
occasionally. Some are written and/or printed in-house; others are produced by 
publishing houses. A brief list appears below. 



Regular Publications 

Research in Higher Education 
New Directions for Institutional 

Research 
Higher Education: Handbook of 

Tlteory and Research 
The AIR Professional File 
Forum Publications 
AIR Newsletter 
AIR Directory 



Monographs 

The Functions of Institutional 

Research 
A Primer on Institutional Research 

Other 

ERIC Collection 



Many of the papers presented at each annual Forum are included in the ERIC 
Collection; some are published in Research in Higher Education or the AIR 
Professional File. 



Suggestions and offers of assistance may be sent to either of the following: 



AIR Publications Board 

Timothy R. Sanford (chair) 
Director, Institutional Research 
University of North Carolina- 
Chapel Hill 
CB 3350, 200 Carr llldg. 
Chapel Hill, NC 27599-3350 

Telephone: (919) 962-3071 
Bitnet II): usford(rr unc 



AIR Executive Office 

Jean C. Chulak 
Administrative Director 
AIR 

314 Stone Building 
Florida State University 
Tallahassee, FL 32306-3038 

Telephone: (904) 644-4470 
Bitnet ID: air@fsu 
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